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Levels of Questioning in Current Elementary Textbooks: 

What the Future Holds 

by: Gregory P. Risner 
Janice I. Nicholson 
Janice G. Myhan 

Critical thinking maintains a prominent position among desired 
learning outcomes in elementary science. Since the textbook method 
is the primary instructional strategy for teaching science, 
students 1 thinking most often parallels the cognitive level (s) of 
the textbooks being used. 

Questioning has been labeled by Hilda Taba as the "most 
influential teaching act," and research verifies the role of 
higher-cognitive questioning in eliciting critical thinking (Andre, 
1979; Doak, 1970; Hunkins, 1968; Redfield and Rousseau, 1981; Taba, 
Levine, and Elzey, 1964) . 

Because teachers use textbooks as the primary source of 
instruction (Smith, 1980, cited by Mechling & Oliver, 1983), it is 
necessary to analyze elementary science textbooks and their 
accompanying resource materials (e.g., published tests) to 
determine their capacity to develop higher-level thinking skills. 
Textbook publishers provide a wide variety of accompanying 
materials such as study guides, chapter and unit tests, posters, 
parent letters and other materials. Many textbook publishers 
include scope and sequence charts which emphasize the fact that the 
text and accompanying materials emphasize divergent, open-ended 
questions which facilitate critical thinking skills. 



ERIC 



Most of the research studies related to questioning have been 
limited to questions contained in textbooks, and teachers 1 manuals. 
The research literature does not contain studies concerning the 
cognitive levels of test items that accompany elementary science 
textbooks. This research study compares the levels of questions 
from science tests accompanying textbooks published in the 1980s to 
those that will be used in schools in the 1990s. 

ANALYZING TEXTBOOK TESTS 

In an attempt to determine the level of questioning according 
to Bloom's (1956) taxonomy, generated by test items that accompany 
selected fifth-grade science textbooks, an analysis of the tests 
provided by three of the most commonly used elementary science 
textbooks was completed. Additionally this analysis examined the 
extent to which each series emphasized the higher levels of 
questioning and what significant differences, if any, existed 
between the old and new publications in the number of knowledge- 
level and above-knowledge-level test items. Moreover, it was 
hypothesized that there would be no significant difference between 
the old and new textbook editions in the distribution of knowledge- 
level and above-knowledge-level test items included in end-of- 
chapter tests. 

PROCEDURES 

Textbook adoption information was obtained from the state 
departments of education of Alabama, California, Tennessee, and 
Texas to obtain a rank ordering of the fifth-grade series used most 
frequently for all states combined. Three of the most widely used 
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textbooks, both old and new editions, published in the 1980s were 
analyzed: Accent on Science (1983) and Me rrill Science H989), 
both published by Charles E. Merrill Company: H olt Science (1984) 
and Holt Science (1989), both published by Holt, Rinehart, and 
Winston Company: and Scott. Foresman Science (1983) and Scott . 
Foresman Science (1989), both published by Scott, Foresman and 
Company . 

One hundred items from end-of -chapter tests were analyzed from 
each edition (old and new) of the textbook series (200 per series) , 
which represented a total sample consisting of 600 items. 
Interrater agreement on training materials reached the 96% level. 
The 600 items were randomly selected and categorized by the 
researchers and one experienced classroom teacher, according to the 
six major categories included in Bloom's (1956) taxonomy: 
knowledge, comprehension, application, analysis, synthesis, and 
evaluation. Results of the classifications were analyzed and 
presented in frequency distribution tables and a series of circle 
graphs. The chi-square statistic was selected to test the proposed 
hypotheses. 

FINDINGS AND CONCLUSION S 
As indicated in Table I, a total of 173 (58%) of the 300 test 
items in the old editions represented the knowledge category. 
These series contained 109 (36%) comprehension items and only 6% 
application items. Test items reflective of the nalysis and 
synthesis categories were nonexistent in the distribution; only one 
evaluation item existed. 
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Among the new editions (see Tab? II), the knowledge category 
received even greater representation collectively. Of the 300 
items analyzed, 209 (70%) items were judged as knowledge, 85 (28%) 
as comprehension, and only 6 items as application. 

This analysis also examined the extent to which the test items 
in both the eld and new editions emphasized the higher levels of 
questioning. For this purpose, levels two through six of Bloom's 
(1956) taxonomy were combined and percentages were calculated. By 
collapsing the upper five levels into one major category (i.e., 
higher level questions) , both new and old textbook editions were 
examined according to their emphasis on understanding or applying 
science information (higher-level) as compared to memorization 
(lower-level) . The resulting dichotomies of the old publications 
as shown in Figures 1-3, indicated that two series, Merrill and 
Holt, possess generally balanced disbursements of lower and higher 
level questions on the published tests, thereby exposing students 
to cognitive stages other than mere recall. Closer examination of 
the higher level questions reveals that Merrill contains more 
application items than Holt, however both series demonstrated a 
preponderance of comprehension items. While providing cognitive 
exercise other than memorization, Merrill and Holt focus primarily 
on the lowest level of understanding. As indicated by Figure 3, 
the older edition published by Scott, Foresman contained the lowest 
percentage of higher level questions. The higher level category in 
this series possessed 17 comprehension items and one application 
item. One obvious conclusion is that chapter-end science tests in 
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this edition predominantly emphasize mere recall of isolated bits 
of scientific information. 

The analysis of: new textbook publications reveal an 
interesting trend with regard to the emphasis of higher level 
questions. As shown in Figures 4-6, recent publications of both 
Merrill and Holt place less eiaphasis on the higher level questions 
that did older editions; the similarities among these new 
publications are striking: Merrill and Holt possessed only 26% and 
30% higher level questions respectively. On the other hand, Scott, 
Foresman demonstrated more higher-level test items in the new 
editions. 

In order to determine if changes had occurred over time, the 
study compared old and new editions for each of these series to 
determine if significant differences existed in the distributions 
of knowledge and above nowledge-level test items. The chi-square 
(X 2 ) statistical technique was employed through a series of 2 x 2 
contingency tables (Textbook Editions: Old and New x Knowledge- 
Level vs. Above-Knowledge Level Questions) . 

Results of these analyses revealed that Merrill contained 
significantly more knowledge-level items in the new editions (X 2 = 
23.98; p < .001) . A similar emphasis on the knowledge category was 
demonstrated by the new Holt science series as indicated by the 
significant (p < .001) chi-square (X 2 ) value of 9.15. The Scott, 
Foresman series however, demonstrated no significant difference (X 2 
= 3.37) in the level of questioning between the old and new 
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editions, indicating that questioning levels in this text remained 
unchanged , 

DISCUSSION 

Based on the assumption that the three science textbooks 
selected for study are representative of all science books, one 
obvious conclusion is that higher order thinking skills are not 
elicited by end-of -chapter tests that accompany science textbooks. 
The lack of emphasis on application, analysis, synthesis, and 
evaluation levels of questioning indicates a gap between the stated 
goals of elementary science textbooks and actual published 
material. It is recommended that science textbook publishers 
include test items that engage students with substantial practice 
with comprehension, application, analysis, synthesis, and 
evaluation questions. Teachers must also devise supplementary 
questions, both written and oral, that facilitate the development 
of higher cognitive processes. 

Similar research in the area of science is needed to determine 
the cognitive levels of questioning utilized in teacher's editions 
for classroom use and pupil's textbooks for chapter-end review. 
Other recent experimental studies are needed to determine the 
effects of higher-level questions on student achievement, attitude, 
and interest in elementary science. 

Prepared by: 

Gregory P. Risner 
Janice I. Nicholson 
Janice G. Myhan 
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TABLE 1 

FREQUENCY AND PERCENTAGE OF END-OF-CHAPTER TEST ITEMS FOR THREE PUBLISHED 
FIFTH-GRADE SCIENCE TEXTBOOK SERIES ACCORDING TO BLOOM'S TAXONOMY 

OF EDUCATIONAL OBJECTIVES 
(OLD EDITIONS 1983-1984) 
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Total 


100 


100 


100 


100 


100 


100 


Note. 


Percentages 
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rounded to the 


nearest whole 


numbers. 
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TABLE 2 



FREQUENCY AND PERCENTAGE OF LuD-OF-CHAPTER TEST ITEMS FOR THREE PUBLISHED 
FIFTH-GRADE SCIENCE TEXTBOOK SERIES ACCORDING TO BLOOM'S TAXONOMY 

0? EDUCATIONAL OBJECTIVES 
(NEW EDITIONS 1989) 
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numbers . 
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Figures 1-3 



Percentage of Knowledge-Level and Above 
Knowledge-Level End-of-Chapter Test Items for 
Three Selected Fifth Grade Science Textbook 

Series 
(Old Editions 1983-84) 
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Figures 4-6 



Percentage of Knowledge-Level and Above 
Knowledge-Level End-of-Chapter Test Items for 
Three Selected Fifth Grade Science Textbook 

Series 
(New Editions 1989) 
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